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ABSTRACT 


The detection of object with respect to vehicle and tracking is the most 
needed step in computer research area as there is wide investment being 
made form intelligent traffic management. Due to changes in vision or 
scenes, detection and tracking of vehicle under different drastic conditions 
has become most challenging process because of the illumination, shadows. 
To overcome this, we propose a method which uses tensorflow fused with 
corner points of the vehicles for detection of vehicle and tracking of an 
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CNN passed to track the detected object, using the tracking algorithm based on 
Rees CNN. The proposed algorithm gives result of 90.88% accuracy of detection 
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ensorFlow 
Tracking 
Vehicle detection This is an open access article under the CC BY-SA license. 
©) BY SA 
Corresponding Author: 
Mallikarjun Anandhalli 


Department of Electronics and Communication Engineering 
KLS Gogte Institute of Technology 

India 

Email: malliarjun7 1 @ gmail.com 


1. INTRODUCTION 

Since from few decades many methods were proposed to detect and track object in foreground, but 
in one or the other way methods couldn’t give accuracy in the challenge [1]. Main aim of the proposed work 
is to find the object position to detect and track it which is used in various fields like military, medical and 
video analysis, usually frame consists of background as well as foreground data [2], here foreground image is 
represented by ROI feature points, and other points are considered as the background. In the surveillance 
system it follows two main procedures those are detection of the moving object and calculation of the 
motion. First is detection of the object and it is carried out by background pixels data. 

Many researchers have developed different methodologies for object detection in video where many 
combinations of methods and differences in protocols exist. Background subtraction is used to segment the 
moving object in the video [3]. 

In reality, the optical flow algorithm is used to overcome this disadvantage but when different 
conditions are considered, sometimes it gives false alarm for algorithms. In about all the background 
subtraction cases, background data is used by object trackers however it leads to mis-classification. Next to 
improve the accuracy of this algorithm, selection of robust classifier is used [4]. 

In the era of increasing population every individual makes use of an automotive industry i,e; the use 
of vehicles has been increased rapidly. This made adverse effect on traffic management system. There are no 
novel approaches or algorithms for the detection of vehicles running on the road to find the density of 
vehicles. All the algorithms proposed till today detect vehicles only in particular climate conditions most of 
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the algorithms doesn’t hold good for all the climate conditions. Hence the proposed algorithm makes use of 
tensorflow in order to detect objects. And later the objects are tracked to stabilize algorithm for detection of 
vehicles. 

To overcome this problem, in this proposed method generalized object detection algorithm based on 
tensorflow and object tracking algorithm based on CNN has been used. These methods are found robustly 
and tracked the objects in the complicated images and background scenes. This is based on this condition that 
the images have strong local two-dimensional structures. The CNN combines three architectural ideas to 
guarantee a difference in variance of change, up to certain degree and distortion: shared weights, local 
receptive fields and sometimes, temporal and spatial sub-sampling. The main problem lies in tracking of the 
object in the field of computer vision. Most of the methods such as, blobs movement [5-10], follow-up 
balanced which is a basic model that is considered over the time. The challenges faced by these techniques in 
real time situations are enormous because: 

— Do not have a model that distinguishes itself from others in the category of interest. 
— The model from its original can deviate over the time. 

Many methods were proposed by researchers’ community from background substitution to CNN. 
Few algorithms have been listed here and discussed. Survey gives objects feature point detection, 
segmentation, classification algorithm and background subtraction. To have exact tracking, features of object 
are needed therefore detection of object has main role here. Probabilistic and deterministic motion model and 
the model based on appearance are adapted to get better accuracy. The process of tracking involves training 
of the feature points and updating. Main problem lies when tracking of the object requires huge features 
which are not possible most of the times [9]. 

To enhance the performance, CNN is utilized for classification and recognition of the image. CNN 
includes lakhs of different images of different class [10-11]. CNN is used to extract information of image and 
to study complicated features. Algorithm for object tracking based on CNN with the architecture of shift 
variant is used to get features we studied. The transitory features are being studied during online process. The 
behaviors such as temporal and spatial are considered by a couple of images in place of an image [12]. 
R. Phadnis et al. [13] did involve approach, where the results of last year of pre-set CNN module are 
cascaded with online SVM to study different appearance systems. Bayesian network targeting is used for 
particular saliency map thus making the algorithm very much complicated for just detecting the vehicles. The 
algorithms work for the given set of datsets under a condition but when considered for different climatic 
condition the accuracy will have major affect. Hence, we have proposed an algorithm which works in all 
given climatic conditions [14-21]. 

The brief introduction and contributions of the model is as listed: (a) A method is put forth to detect 
vehicles based on tensorflow and corner points of the vehicle and double verifying the detected object with 
upholding the accuracy of the proposal. (b) The proposed model use corner points which help to detect the 
object anywhere irrespective of the climatic changes. 


2. RESEARCH METHOD 

Architecture of the proposed system is shown in Figure |. From the architecture one can make out 
that vehicle detection is performed using tensorflow. Once the vehicles are detected the corner points are 
detected within the vehicle region. The detected corner points help in verifying whether the detected object is 
vehicle or no. After the detection of the vehicles, the interested points from the vehicular region are passed to 
CNN algorithm which helps in tracking. 


Vehicle Detection Corner Points Vehicle Tracking 
Using Tensorflow using CNN 


Figure 1. Flow Architecture of the proposed algorithm 


In this system video is considered as an input. In the next step frames are considered for processing. 
Deep learning methods are used in detection of object and it’s tracking algorithms. The detection of object is 
explained in depth in different conditions like light variation, illumination using tensorflow [22]. 

The object detection API based on tensorflow is a platform of open source, it is easy to build, train 
and detect and it is built on top of tensorflow. This detection of object which is based on tensorflow is 
proposed in Figure 2. In proposed method firstly the libraries which are necessary are imported. Later the 
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system which is pre-trained for object detection is imported, along with box and tensor class weights are 
initialized. 

After the initialization of all the parameters of the tensorflow model, the detected object in the 
image is taken. The tensorflow is used to detect vehicles, once the detection of vehicles is done using 
tensorflow then the corners present the image are calculated which will help us in double verification of the 
detected vehicles. The Figure 2 shows the flow chart of the model. 


Finding 
location of 
object? 


Figure 2. Flow diagram of the proposed algorithm 


The tensorflow model which is loaded from the trained model is applied on the image, the image is 
tested by this model and then the object’s location in the image is returned. This method is based on the 
algorithm of tensorflow object detection. The accuracy of this method is better and can be applied to RGB 
images. To initiate the tracking process objects locations are needed. In this method CNN based tracking 
algorithm is used instead of convolutional computer vision-based algorithm. 


2.1. Corner points 

From the detailed analysis of vehicular region in an image after the corners are detected it is 
found that the corners are densely packed with in the vehicular region and can be seen in the following 
Figure 3(a-b). Once the detected vehicles are verified then the centroid of the object or the interest points of 
the objects are passed to the CNN. The CNN will help in reliable tracking of the vehicles even though the 
occlusion, shadows appear in the image [23-24]. 

The A; and 42 are the Eigenvalues which are calculated from the matrix M computed from image 
derivatives I, and I, for the detection of the corner points. Based on these Eigenvalues pixels are classified 
and they are: 

1. IfA, >> A, or A1 >> A2 Then the pixel is edge 
2. IfA; and A2 are larger than it is corner 

3. IfA; and 2 are small than it is flat 

The corner response is calculated by 


R = detM — k(trace M)? 


det M = A1A2 
trace M = A1&A2 
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Where k is constant and varies from 0.04 to 0.06 
Response is large for a corner and it only depends on M 


(b) 


Figure 3. Results of corner detection, (a) Original input file, (b) Output of corner detection 


2.2. Grouping of corner points 

The comer points that lie in the non-vehicular region are removed as they lie in non region of 
interest. The points are densely packed within the region of interest hence these points are grouped using 
Euclidian distance and is given as 


f= V2 - x)? + 02-1)? 


Before grouping of points a threshold value is set. This value is nothing but the maximum distance 
between any two points in an image to group them as one and also each group should have minimum number 
of corner points to consider the object has vehicle. The threshold value varies with respect to the resolution of 
videos, and camera. If the distance between the points is less than the threshold, they are discarded. In this 
way corner points are grouped accordingly which in turn helps in re-verification of the object. 

If the distance between any two points is less than the threshold value set the two points are grouped 
as one. In this way, set of random interest points are grouped based on the threshold value. Each group of 
points correlates with that of a vehicle. 


2.3. CNN 

The algorithm related to the tracking of the object plays a very important role in computer vision 
area. Understanding of homogenous motion and analyzing variation caused over time and information of the 
object are needed for accurate tracking of an object. Tracker should be able to adopt for any new 
observations and model them accordingly. The model is trained with the pre-defined weights. The model has 
the ability to include the temporal information. The pre-set model which is trained on huge number of objects 
in real time is used instead of concentrating on the objects during testing time. The object at the speed of 
150fps can be tracked by these lightweight systems. Also, it has ability to remove the occlusion problem. 
And in the proposed work, the locations of the objects which are achieved from the algorithm of object 
detection based on tensorflow are passed to the algorithm of object tracking based on CNN. First the 
positions are studied by the model and are searched within the frames. 

Although CNNs use max-pooling layers to allow CNNs to be spatially-invariant for the data that is 
being given as the input, the feature maps which are intermediate are not actually invariant to the 
transformations which are large of the input data. Hence, we not only classify the patch into object but also 
classification of the location of the object in the patch or basically tracking is accomplished. Instead of 
generating random patches that cover the region of interest (ROD, here we generate fixed-spacing random 
patches that cover the ROI. It also reduces the risk of missing the target. Here, instead of passing each patch 
to the convolutional layer for classification, the whole ROI is passed through the layer, which would save the 
redundant computations. Taking the mean location of the patches the target location is determined. Bilinear 
interpolation can be used for object detection applications. 
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3. RESULTS AND DISCUSSION 

In normal situations, all the state of art algorithm has shown good results but in other environmental 
conditions like rain, fog, snow it is difficult to get accurate and good results. It may fail to either to detect 
vehicle or track the vehicle and hence motion calculation of the object leads to varying results. The proposed 
algorithm is effective even in complex conditions such as trivial condition such as shadows. The reason for 
this is due to the corner features which are shift and rotational invaraiant. Even under less illumination the 
corner points are detected. The qualitative results were obtained through the evaluation process of ground 
truth image and algorithm output files as input hence result can be accurate and average. The testing of 
algorithm is done on different videos in different environmental conditions. The datsets were collected from 
web. 

The results of algorithm shown in Figure 4(a-b) are sunny with low resolution videos even though 
the tensflow miss the detection of the algorithm, but the corer points help in detection of the objects as 
vehicles. The detection rate of the tensorflow is 90% on the standard datasets. But whereas the detection rate 
of the proposed algorithm is almost equal to 90%, the tabular column defines these parameters. 


Figure 4. Results of vehicle detection under low resolution sunny day, (a) First example, (b) Second example 


The results shown in the Figure 5(a-b) are cloudy and sunny with good resolution videos and can be 
seen that the algorithm detects all of the vehicles irrespective of the distance. On glance the Figure 5(b) and 
Figure 4(b) one can make out that the cyclist is not detected as a vehicle. The main reason for this is that 
vehicles do pose a good number of corner points and where as non vehicle region do not. This analysis of the 
corner points have really helped to achieve higher accuracy on all types of datasets. 


(a) 


Figure 5. Results of vehicle detection under different climatic condition, (a) Cloudy Condition, (b) Sunny day 


The results shown in the Figure 6(a-b) are rainy low and good resolution videos and can be seen that 
the algorithm detects all of the vehicles irrespective of the distance. The basic property of the ir-rotational, 
shift-invariant due to these properties the vehicles can be detected even in shadow regions and rainy regions. 
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The reason is the corner points pose under any of these conditions and thus helping in detection of the 
vehicles. 


Figure 6. Results of vehicle detection under different climatic condition, (a) Rainy condition, (b) Shadow 
condition 


Based on result analysis made considering different environmental scenario the complete anaylsis 
has been put in the Table 1. In the Table 1, the resulotion of the corresponding dataset has been considered 
for the study along with FPS, Climate, No. of vehicles present vs vehicles detected along with missed 
detection and false detections as been listed in the Table 1 along with the corresponding results. 


Table 1. Results of proposed model 


F ; Vehicles No. of vehicles Missed Falsely detected 
ore: Bootes, Ep ee Detected Present (TP) vehicles (FN) vehicles (FP) 
1 320*240 10 Sunny(low resolution) 4 4 0 0 
2 320*240 15 Sunny(low resolution) 5 0 0 
3 640*480 25 Cloudy 8 8 0 0 
4 320*240 30 Sunny (High resolution video) 4 4 0 0 
S| 704*480 30 Rainy 8 8 0 0 
6 704*480 30 Sunny Complex (Multiple) 5 4 0 1 


3.1. Performance 

The performance of the model is compared with state of art systems. The parameters to measure the 
performance of the model are correctness; completeness and quality which are explained by Wiedemann and 
are listed is [10]: 
True positive (TP): Vehicles accurately detected as vehicles. 
False positive (FP): Vehicles incorrectly found as vehicles. 
False pegative (FN): Vehicles wrongly identified as non-vehicles. 


TP 
TP+FP 


TP 
TP+FN 


*100% Quality =" —*100% 


*100% Completeness = TP+FN4FP 


Correctness = 


3.2. Computation effectiveness 
Opencv [19] tool along with C++ is used to implement the algorithm. The computational cost of the 
system is 105ms to process a frame which is considerable in real world scenario. 


3.3. Comparative analysis 

The states of art algorithm [1, 25-26] are considered for the comparison with the model based on the 
results obtained. The result of the algorithm is more reliable and is simple, effective and time efficient. 
Accuracy of the system is 92% in multi view, in side view and top view the algorithm is even better with 
92% because we can easily differentiate the colors with the surrounding environment, in front and rear view 
the accuracy is 92% which is better than the other results and can hold for all the views. On comparing the 
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accuracy of proposed algorithm with other three state of art works with respect to there detection view the 
proposed algorithm has 92% accuracy in detection of the vehicle as shown in Table 2. 


Table 2. Comparison of the results 
View Accuracy 


Algorithm Side View and Top View ‘Front and Rear View = Multi View 


Proposed Algorithm 97% 91% 92% 
Combining Multiple Parts for Complex Urban Surveillance [1] Xx 92% x 
Morphological operations [26] 91.5% x Xx 
Low-Exposure Color Video for Night Conditions [25] 89% Xx x 


4. CONCLUSION 

In proposed paper, tensorflow fused with comer points are taken into the consideration for the 
detection of the objects as vehicles. From the region of Interest, the points are passed to the convolutional 
neural network which in turn is used for vehicle tracking. The proposed algorithm for the vehicle detection 
detects all the vehicles present in the any of the scenario under different climatic conditions and the results in 
terms of its accuracy and time taken are very good. The tracking algorithm based on CNN is used for 
tracking the detected vehicle. The parameters which were set for the proposed algorithm, which is the 
sensitivity with detection rate of 92%, similarly accuracy of about 91.8%, specificity of about 91.2%. 
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